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Tables have been prepared for use in any experiment designed to selec 
that particular one of k binomial processes or populations with the highest 
{long time) yield or the highest probability of siiccess. Before experimenta- 
tion the experimenter chooses two co7istants d* and P* (0 < d* ^ 1; ^ 
P* < I) and specifies that he would like to guarantee a probability of at 
least P* of a correct selection whenever the true difference between the long- 
time yields associated with the best and the second best processes is at least 
d*. The tables show the smallest number of units required per process to be 
put on test to satisfy this specification. Separate tables are given for k = 2, 
3, 4 and 10. Each table gives the result for d* = 0.05 (0.05) 0.50 and for 
P* = 0.50, 0.60, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95, and 0.99. For values of 
d* and P* not considered in the tables, graphs are given on which inter- 
polation can be carried out. Graphs hare also been constructed to make 
possible an interpolation or extrapolation for other values of k. An alterna- 
tive specification is given for use when the experimenter has some a priori 
knowledge of the processes and their probabilities of success. This specifica- 
tion is then compared with the original specification. Applications of these 
tables to different types of problems are considered. 

INTRODUCTION AND SUM\L\KY 

A frequently encountered problem is that of selecting the "best" 
one of k (k ^ 2) processes or populations on the basis of the same num- 
ber n of observations from each process. We shall assume that the given 
processes are all binomial oi' "go^no go" processes and that the best 
process is the one with the highest probability of obtaining a "success" 
on a single obser\'ation. We shall consider a single sample or nonse- 
quential procedure which means that the common number n of observa- 
tions from each process is to be determined before experimentation 
starts. The corresponding sef|uential problem is being investigated.' 
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Briefly, the technique employed here is to let the experimenter decide 
how '■'close" the best and second best processes can be before he is willing 
to relax his control on the piolmbility of a correct selection. The selection 
of a best process will, of course, be made on the basis of the largest 
observed frei]uenf.y of "success"; the only remaining problem is to 
determine the value of n. Tables and graphs which cover almost all 
practical problems in this framework are given for determining the re- 
quired value of n. In particular, tables and graphs are given for k — 2, 
3, 4 and 10. Graphs are also given to approximate the result for any value 
of k up to 100. 

This problem arises in many widely different fields of endeavor; we 
shall briefly consider two industrial applications. One application of the 
binomial problem is to comparative yield studies. Here success corre- 
sponds to the making of a good unit, and the goal is to select the process 
with the highest (long-time) yield. Another application of the binomial 
problem is to comparative life testing studies. In this case the experi- 
menter selects a fixed time T and defines the best process as the one for 
which the probability of any one unit surviving this time T is highest. 
Then, of course, a successful unit is one which survives the time T. In 
treating this as a binomial we are discarding the information contained 
in the exact times of failure. In many cases the times of failure are either 
unknown or very inexact; in other cases it is not kno^vni how to utilize 
the knowledge of the exact times of failure. Hence, it would be valuable 
to know the results for the more basic binomial problem. The time T is 
considered fixed throughout; its value is determined by non-statistical 
considerations. The specification and the final decision of the experi- 
menter all refer to this predetermined time T. It should be noted that 
the experimenter (.-annot use information obtained from the continuation 
of the test beyond time T since the best process for T is not necessarily 
the best process for a longer time, say IQT. This binomial type of analysis 
has the advantage that it does not assume any particular form of the 
life distribution. In particular, the assumption of exponential life is 
avoided. 

The presence of a priori information changes the number of observa- 
tions required. An alternative specification is given which is justified by 
certain k priori information based on past experimentation. The amount 
of saving is briefly examined. This area of utilizing k priori information 
to reduce the number of observations required should be investigated 
further. 

The treatment of the problem in this paper is based on the assump- 
tion that, after experimentation is carried out, the experimenter must 
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choose one of the k procefsses and assert that it is best. If he allows him- 
self the possibility of hedging and asserting that he needs further ex- 
perimentation, then the problem changes and the tables of this paper are 
not appropriate. 

The following additional assumptions will be made: 

1. Observations from the same or different processes are independent. 

2. Observations from the same process have a common fixed proba- 
bility of "success". 

3. There is no chance of error in determining whether a success or a 
failure has occurred. 

The assumption of a common probabiHty fixed once and for all for 
each process is one that should be checked carefully in any practical 
apphcation of the results in this paper. Roughly speaking, this assump- 
tion states that each of the processes is in a state of statistical control 
as far as the probability of success is concerned. 

We shall consider only the case in which the same number n of obser- 
vations are taken from each process. This is certainly reasonable for a 
single sample procedure if no ^ priori information is assumed. 

STATISTICAL FORMULATION OF THE PROBLEM 

Each of k given binomial populations !!,■ is associated with a fixed 
probability of success p,- where ^ p,- g 1 (i = 1, 2, ■ ■ ■ , A'). For ex- 
ample, in the yield problem p, is the long-time yield for process 11, oi' 
the probability that any one unit from IT, is a good one. Let the ordered 
\'alucs of the pi be denoted by 

jO[ll ^ ^"121 ^ ■ •• ^ Plk] (1) 

No a priori information is assumed about the values of the p, or about 
the correspondence between the ordered p[,] and the k identifiable popu- 
lations n, . In particular, we ha\-e no idea before experimentation 
starts whether p|i| is associated with 111 , 1X2 , ■ ■ ■ , or IIi . 

The problem is to select the population associated with p[i) on the 
basis of n observations from each population. If there are t ties for first 
place, say 

p[i] ^ Pm = ■•• = p[i] > Pit+i] (t- < k) (2) 

then we shall certainly be content with the selection of any one of the 
associated t populations as the l)est one. 

As an index of the true difference (or distance) between the best and 
second best populations we introduce the symbol 

<i = P[i] - P[2] (3) 
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It is assumed that if the difference d between the best and second best 
populations is small enough, then the error involved in wrongly select- 
ing the second best process as the best one is an error of little or no con- 
sequence. The experimenter is therefore asked to specify two quantities 
which will determine the number n of observations he is required to 
take from each process. 

Specification : He specifies the smallest value d* 
(0 < rf* ^ 1) of rf for which it \vould be economically 
desirable to make the correct selection. He also specifies (4) 
a probability P* {0 ^ P* < 1) of making a correct 
selection that he ^vould like to guarantee whenever the 
true difference d ^ d*. 

Letting Pes = Pes ivm > ■ ■ ■ > Vw) denote the probability of a correct 
selection we can now rewrite the specification that the experimenter 
wants to satisfy in the simple form 

P^s ^ P* foi- d'^ d* (5) 

[The word "specification" will be used below to denote the specified 
pair of constants {d*, P*) as well as the condition (5) ; it will be clear 
from the text which is meant.] Since the final selection is to be made on 
the basis of the observed frequency of success, the essential problem is 
to find the number n of observations required per process to satisfy the 
specification (5). 

The possibility that d may be less than d* is not being overlooked. 
The region d < d* is being regarded as a zone of indifference in the 
sense that if d < (/*, then we do not care which process is selected as best 
so long as its p-value is within d* of the highest p-value p^^ . For values 
of P* ^ l/A- no tables are needed since a probability of 1/k can be at- 
tained by chance alone. 

Some comments on the above approach and on a possible modifica- 
tion have been placed in Appendix I in order to preserve the conti- 
nuity of the paper. 

CONFIDENCE STATEMENT 

After the experiment is completed and the selection of a best process 
is made, the experimenter can make a confidence statement with confi- 
dence level P*. Let p, denote the true p-value of the selected population 
and let p^ denote the maximum true p-value over all unselected popula- 
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tions. Thou the confidence statement, consisting of two sets of inequali- 
ties 

Pii] - f/* ^ ]h S p\i]] fO ^ /j|i| - p. ^ (/*' 

iPi2i ^ p.. ^ pi.i + <n [o s p. - pi..i ^ (/*, 

has confidence level P*, It should lie noted that the above confidence 
statement is not a statement about the value of any p I)ut is a statement 
about the correctness of the selection made. 

LEAST FAVORABLE CONFIGUHATIOX 

The main idea used in the construction of the tables was that of a 
least favorable configuration. Before defining this concept we shall 
define the set of configurations 

Pill - (/ - P[2] = P[3] = ■■■ = pn-i (6) 

obtained by letting d in ((i) \'ary o\'er the closed interval (d*, 1) as the 
Less- Favorable set of configurations. It is intuitively dear and wiU be 
rigorously shown in Appendix II that if our procedure satisfies the 

specification for any true configui-ation (6) with d = (/' and p|]] = p^i], 
then it will also satisfy the specification when 

pti] - d^ fe Vm § V{2\ ^ ■ ■ ■ ^ P(.(-j (7) 

Of course, we shall be interested particularly in the case in which d 
equals the specified value (/*. If d = d* is fixed in {(i), then (6) specifies 
the difTeronces between the ?j-\'a!ues, hut the "location" of the set is still 
not specified. We shall use p\i] to locate the set of 2:>-\'alues. The proba- 
bility Pes of a correct selection for configurations like (fi) with d = (/* 
depends not only on d*, n and /,■ but also on the location pm of the 
largest p-value (except for the special case /; = 2 and n = 1). [In the 
corresponding problem for selecting the largest population mean of A- 
independent Normal distributions with unit \ariance,' this probability 
Pes depends onlij on the dijfc.rcnces and, hence, only on d in the configura- 
tion corresponding to (fi)]. 

When (0) holds with any Ji. red value of d, the probability Pc.h (for any 
fixed 7;) may be regarded as a function of p^] where d ^ pm ^ 1). This 
function is continuous and l>ounded over a closed interval and therefore 
assumes its mininmm \-alue at some point piu (d) = p^] (rf;/i) in the 
clo.sed interval (d,\). Fig. 1(b) gives the value of pm (d) as a function 
of d for k = 3 and for n = 1, 2, 4, 10 and x . For any particular value 
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of n and for d = d* we shail be particularly interested in the value ptn = 
Pw {d*;n) since this (as shown in Appendix II) gives the smallest 
probability Pes of a correct selection for all flic configurations included 
in the statement of the experimenter's specification. This particular con- 
figuration (6) with d = d* and p\i] = p'[\] (which depends on n) is 
called the Least- Favorable Configuration. 

Although the least favorable configuration tlepends on n, it has been 
empirically found that for n ^ 10 (and in some cases for n ^ 4) the least 
favorable configuration is approximately given by pfi] ^ | (1 + d*) 
in which the two values, pni and p[i\ = p'{i\ - d* are symmetric about ^. 
This symmetric configuration clearly does not depend on n. Fig. 1(b) 
shows that as n — > ^ the least favorable configuration approaches this 
symmetric configuration (i.e., the straight line marked n = x) quite 
rapidly for any value of (/. In Appendix III it is proved that the sym- 
metric configuration is least favorable as ?i — > ^. Fig. 1(a) shows for 
k ^ 3,71 = 10, and any value of d the error in Pes which arises as a result 
of using the symmetric configuration instead of the true least favorable 
configuration. 



0,0002 




Fig_ 1 _ (a) Error in Peg as a result of using the symmetric configuration in- 
stead of the least favorable configuration for /; = 3, ?i = 10, and any common true 
differenceri. (b) Least favorable value pu) (d) oi pu] as a function of the common 
true difference d = pd] - p[«] ,i ^ 2, for A- = 3 and selected values of n. (for 

d = d*, pm (d) = pu]) 
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CONSTRUCTION OF THE TABLES 

Consider any fixed value of (/*. ¥ov each of a set of increasing values 
of n the minimum probability Pt-.t of a correct selection for d ^ d* 
(i.e., the prol)ability for the lea.si favorable configuration) was computed. 
These calculations were thou inverted to find the smallest 71 for which 
the Pes is greater than or e(iual to the specified value P*. Tables I 
through IX give the smallest value of n for k = 2, 3, 4, and 10, for (/* = 
0.05 (O.Oo) 0.50, and for selected values of P*. Graphs corresponding to 
these tables are given in Figs. 2 through 5. 

For small \'alues of u (say, n < 10) it was necessary to approximate 
Pin by calculating the P,s cxacthj for several values of pm and proceed- 
ing in the direction of the minimum probability Pea- For the special 
case n = 2 and /,: ^ 3 an explicit formula for p[\] is given on Fig. 1. 

For large values of n (say, n > 10) the P^-s was calculated by assum- 
ing the symmetric configuration. Here it was necessary to make use of 
the normal approximation to the liinomial. Fortunately the appropriate 
(able needed in this normal approximation is already published.^ The 
proof that this table is appropriate is given in Appendix III. The result- 
ing value of n is gi\'en by 

- - ■ « = ^a-^^*') = ^. (8) 

where the constant B, depending on P* and k, is equal to \C^ and C is the 
entry in the appropriate column of Table I of H. E. Bechhofer's paper.^ 
A short table of B values, Table V (see page 550), is included in this 
paper to make it self-contained. 

The middle expression in (8) will be referred to as the normal ap- 
proximation and the right hand expi'ession in (8) will be referred to as 
the "straight line" approximation. In many cases it has been empiri- 
callij found that these two expressions give close lower and upper bounds 
to the true value. Thus by noting the curves di'awn in Figs. 4 and 6 for 
/■ = 4, P* ^ 0.75 it appears that for all A'alues of d* the true Pes is 
between the normal approximation and the straight fine approximation. 
A.ssuming this to bo so, it follows that for k ^ 4, P* ^ 0.75 the required 
value of 11 satisfies the incquahties 



:^ (1 - <^*"') 



Lr/*d 



^n^\^\ (9) 



where [x\ denotes the smallest integer greater than or equal to the en- 
closed quantity .r. This result (9) is empirical and not based on any 
mathematically proven inequafities. It is used here only to estimate the 
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Table I — Number of Units Required per Process to Guarantee 
A Probability of P* of Selecting the Best of k Binomial Proc- 
esses when the True Difference pm — p^] is at Least d*. 
{k = 2) 
The three ^-allies in eat^h group are: (1) Normal approximation, (2) 

Straight line approximation, and (3) Smallest integer required. 



d* 


P* 


0,50 


0.60 


0.75 


0.80 


0.85 


O.'JO 


0,95 


0.99 


0.05 







12.81 

12.84 
14 


90.77 
90.99 

92 


141.30 
141.66 
142 


214.29 
214.83 
215 


327.66 
328.48 

329 


539.77 
541 . 12 

541 


1079.70 
1082.41 

1082 


0.10 







3.18 
3.21 
4 


22.52 

22.75- 

23 


35,06 
35.41 
36 


53.17 
53.71 
54 


81.30 

82.12 
83 


133.93 
135,28 
135 


267.90 
270.60 
270 


0.15 







1.39 
1.43 
2 


9.88 
10.11 
11 


15.39 

15.74 
16 


23.33 
23.87 
24 


35.68 

36,50- 
37 


58.78 
60,12 
60 


117,57 
120,27 
120 


0.20 







0.77 
0.80 
1 


5.46 
5.69 
6 


8.50- 

8.85+ 
9 


12.89 
13.43 
14 


19.71 
20,53 
21 


32.47 
33.82 
34 


64.94 

67.65+ 

67 


0.25 







0.48 
0.51 
1 


3.41 
3.64 
4 


5.31 

5.67 
6 


8.06 
8.59 
9 


12.32 
13.14 
14 


20.29 
21.64 
22 


40,59 
43.30 
42 


0.30 








0.32 
0.36 
1 


2.30 
2.53 

3 


3.58 

3.93 
4 


5.43 

5,97 
6 


8.30 
9,12 

9 


13.68 
15.03 

15 


27.36 
30.07 

29 


0.35 







0.23 
0.26 
1 


1.63 
1,86 
2 


2.54 
2.89 
3 


3.85- 

4.38 

5 


5.88 
6.70 

7 


9.69 
11.04 
11 


19.38 
22.09 

21 


0.40 







0.17 
0.20 
1 


1.19 

1.42 
2 


1.86 
2.21 
3 


2.82 
3,36 
4 


4,31 
5.13 
5 


7.10 
8.46 
9 


14.21 
16.91 
16 


0.45 







0.13 
0.16 
1 


0.90 
1.12 
2 


1.39 

1.75- 

2 


2.11 

2.65+ 

3 


3.23 
4.06 
4 


5.33 
6.68 

7 


10.65+ 

13.36 

13 


0.50 







0.10 
0.13 

1 


0.68 

0,91 
1 


1.06 
1.42 
2 


1.61 

2.15- 

3 


2.46 
3.28 

4 


4.06 
5.41 
5 


8.12 
10.82 
10 
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Table II — Xumbeu of Units Required per Process to Guaran- 
tee A Probability of P* of Selecting the Best of k Binomial 
Processks when the True DIFFERE^X'E pu] — p[2] is at Least 

(/*. (k = ;{) 

The three ^'alues in each group are: (1) Xorraal approximation, (2) 
Straight Hue approximation, and (3) Smallest integer required. 



d* 


p. 




0.50 


0.60 


0.7S 


0.80 


0.85 


0.90 


0.95 


0.99 


0.05 


30.89 
30.97 

31 


78.16 

78.36 
79 


205.06 
205.58 
206 


272.36 
273.04 

273 


363.06 
363.97 
364 


496.14 
497.38 

498 


732.63 
734.46 
735 


1305.21 

1308.49 
1308 


0.10 


7.66 
7.74 
8 


19.39 
19.59 
20 


50.88 
51.39 
52 


67.58 
68.26 
69 


90.08 
90.99 
91 


123.10 
124.34 
125 


181.78 
183.62 

184 


323.85+ 

327.12 

327 


0.15 


3.36 
3.44 
4 


8.51 
8.71 
9 


22.33 

22.84 
23 


29.66 
30.34 
31 


39.53 

40.44 
41 


54.02 
55.26 
55 


79.77 

81.61 
82 


142.12 
145.39 
145 


0.20 


1.86 
1.94 
3 


4.70 
4.90 
5 


12.33 

12.85- 
13 


16.38 
17.07 

17 


21.84 

22.75- 

23 


29.84 
31.09 
31 


44.07 
45.90 
46 


78.51 
81.78 
81 


0.25 


1.16 
1.24 
2 


2.94 
3.13 
4 


7.71 
8.22 
9 


10.24 
10.92 
11 


13.65- 

14.56 

15 


18.65+ 

19.90 

20 


27.54 
29.38 
29 


49.07 
52.34 
52 


0.30 


0.78 

0.86 
2 


1.98 

2.18 
3 


5.20 
5.71 
6 


6.90 

7.58 
S 


9.20 
10.11 
10 


12.57 

13.82 
14 


18.57 
20.40 
20 


33.08 
36.35- 

35 


0.35 


0.55+ 

0.63 

2 


1.40 
1.60 
2 


3.68 
4.20 
5 


4.89 
5.57 
6 


6.52 
7.43 

8 


8.91 
10.15 
10 


13.15+ 

14.99 

15 


23.43 
26.70 
26 


0.40 


0.41 
0.48 
1 


1.03 
1.22 
2 


2.70 
3.21 
4 


3.58 
4.27 
5 


4.78 
5.69 
6 


6.53 

7.77 
8 


9.64 

11.48 
11 


17.17 

20.45- 

20 


0.45 


0.30 

0.38 
1 


0.77 

0.97 
2 


2.02 
2.54 

3 


2.69 
3.37 

4 


3.58 

4.49 
5 


4.90 

6.14 
6 


7.23 

9.07 
9 


12.88 

16.15+ 
15 


0.50 


0.23 
0.31 
1 


0.59 
0.78 
2 


1.54 
2.06 
3 


2.05- 

2.73 

3 


2.73 
3.64 
4 


3.73 
4.97 
5 


5.51 
7.34 
7 


9.81 
13.08 
12 
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Table III — Number of Units Required per Process to Guar- 
antee A Probability of P* of selecting the Best of /.■ Binomial 
Processes when the True Difference pm — pi?] is at Least 
d*. (fc = 4) 
The three vakics in eacli group iire : (1) Normal approximation, (2) 

Straight hne approximation, and (3) Smallest integer required. 



i' 


P* 


0.50 


0.60 


0.7S 


cao 


0.85 


0.90 


0.9S 


0.99 


0.05 


69.85- 

70.02 
71 


132.65+ 

132.99 
134 


282.27 

282,98 
283 


357.52 

35S.42 
359 


456.82 

457.96 
458 


599.53 
601.03 
601 


848.30 

850.42 
850 


1438.12 

1441.72 
1442 


0.10 


17.33 
17.51 
18 


32.91 

33.25- 

34 


70.04 
70.74 
71 


88.71 
89.61 
90 


113.35- 

114.49 

114 


148.76 
150.26 
150 


210.48 
212.61 
212 


356.83 
360.43 
360 


0.15 


7.61 
7.78 
8 


14.44 
14.78 
15 


30.74 

31.44 
32 


38.93 
39.82 
40 


49.74 
50.88 
51 


65.29 
66.78 
67 


92.37 
94.49 
94 


156.61 
160.19 
160 


0.20 


4.20 

4.38 
5 


7.98 
8.31 
9 


16.98 
17.69 

18 


21.51 
22.40 

23 


27.48 
28.62 
29 


36.06 
37.56 

38 


51.03 
53.15+ 

53 


86.50+ 

90.12 

89 


0.25 


2.63 

2.80 
3 


4.99 
5.32 
6 


10.61 
11.32 
12 


13.44 
14.34 
14 


17.17 
18.32 
18 


22.54 
24.04 
24 


31.89 
34.02 
34 


54.06 
57.67 
57 


0.30 


1.77 

1.95- 
3 


3.36 

3.69 
4 


7.15+ 

7.86 

8 


9.06 

9.96 
10 


11.58 
12.72 
13 


15.19 
16.70 

17 


21.50- 

23.62 

23 


36.44 

40.05- 
39 


0.35 


1.25+ 

1.43 

2 


2.38 
2.71 
3 


5.07 
5.77 
6 


6.42 
7.31 

7 


8.20 

9.35- 

9 


10.76 
12.27 
12 


15.23 
17.36 

17 


25.82 
29.42 
28 


0.40 


0.92 
1.09 
2 


1.75- 

2.08 

3 


3.71 
4.42 
5 


4.70 
5.60 
6 


6.01 
7.16 

7 


7.89 
9.39 
9 


11.16 
13.29 

13 


18.92 
22.53 

21 


0.45 


0.69 

0.86 
2 


1.31 
1.64 
2 


2.79 
3.49 

4 


3.53 

4.42 
5 


4.51 

5.65+ 

6 


5.92 
7.42 

7 


8.37 
10.51 

10 


14.19 

17.80 
17 


0.50 


0.53 
0.70 

2 


1.00 
1.33 
2 


2.12 
2.83 
3 


2.69 
3.58 

4 


3.43 
4.58 
5 


4.51 
6.01 
6 


6.38 

8.50+ 

8 


10.81 
14.42 
13 
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Table IV — Number of Units Required per Process to Guaran- 
tee A Probability of P* of Selecting the Best of h Binomial 
Processes when the Tri'e DiFPERENrE pi,] — pp) is at Least 

d*. {k = 10) 

-The three values in each group are: (1) Normal approximation, (2) 

Straight line approximation, and (3) Smallest integer required. 



d* 


P' 




0.50 


0.60 


0.75 


o.go 


0.85 


0.90 


0.95 


0.99 


0.05 


216.96 

217.50+ 

218 


312.51 

313.29 
314 


511.15+ 

512.43 
513 


604.04 

605.55+ 

606 


722.50- 

724.31 

725 


887.54 

889.77 
890 


1165.49 
1168.41 
1169 


1798.01 
1802.51 
1803 


0.10 


53.83 
54.38 
55 


77.54 
78.32 

79 


126.83 
128.11 
128 


149.87 
151.39 

151 


179.27 
181.08 
181 


220.22 
222.44 

222 


289 . 18 
292.10 
291 


446.12 
450.63 

449 


0.15 


23.62 
24.17 
25 


34.03 
34.81 
35 


55.66 
56.94 
57 


65.77 
67.28 

67 


78.67 
80.48 
80 


96.64 

98.86 
98 


126.90 
129.82 
129 


195.77 
200.28 
198 


0.20 


13.05+ 

13.59 

14 


18.80 

19.58 
20 


30.75- 

32.03 

32 


36.33 

37.85- 
38 


43.46 
45.27 
45 


53.39 
55.61 
55 


70.10 
73.03 

72 


108.15 
112.66 

111 


0.25 


8.16 
8.70 
9 


11.75- 

12.53 
13 


19.22 

20.50- 
20 


22.71 
24.22 

24 


27.16 

28.97 
29 


33.37 
35.59 
35 


43.82 

46.74 
46 


67.59 

72.10 
70 


0.30 


5.50- 
6.04 

7 


7.92 
8.70 
9 


12.95+ 

14.23 

14 


15.31 

16.82 
17 


18.31 
20.12 
20 


22.49 

24.72 
24 


29.53 
32.46 
32 


45.56 
50.07 

48 


0.35 


3.90 
4.44 
5 


5.61 
6.39 

7 


9.18 
10.46 
11 


10.84 
12.36 
13 


12.97 
14.78 
15 


15.93 
18.16 

18 


20.92 

23.85- 

23 


32.28 
36.79 
35 


0.40 


2.85+ 

3.40 

4 


4.11 
4.90 
5 


6.73 
8.01 

8 


7.95- 
9.46 
10 


9.51 
11.32 
11 


11.68 
13.90 

13 


15.34 
18.26 

17 


23.66 
28.16 

26 


0.45 


2.14 
2.69 
3 


3.08 
3.87 
4 


5.05- 

6.33 

6 


5.96 

7.48 
8 


7.13 

8.94 
9 


8.76 
10.98 
11 


11.50+ 

14.42 

14 


17.75- 
22.25+ 
20 


0.50 


1.63 
2.18 
3 


2.35- 
3.13 


3.84 

5.12 
5 


4.54 

6.06 
6 


5.43 
7.24 

7 


6.67 

8.90 
9 


8.76 
11.68 

11 


13.52 
18.03 

16 
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Fig 2. — Number of units n required per process to guarantee a probiibility 
of /-■* of selecting the better of two l)inomiaI processes wlieii the true difference 
d is at least d*. 
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Fig. 3. — Number of units n required per process to guarantee a probability of 
p* of selecting the best of three binomial processes when the true difference d is 
at least d*. 
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Fig. 4 — Number of units n required per process to guarantee a probability of 

P* of selecting the best of four bionimal processes when the true dilTerence d is 
at lenst d*. 
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Fig. 5 — Number of units n reqiiirecl per process to guarantee a probability of 
P* of selecting the beat of ten binomial ])rocesses when the true difference rf is 
at least d*. 
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Fig. 6 — Bounds for the number of units n required per process to guarantee a 
probability of P* of selecting the best of four binomial processes when the true 
difference d is at least d". 



Table V — Values of B = 10 to be Used with the Normal 

Approximation (8) where C is Obtained from Table I 

OF R. E. Bechhofer's Papers 



Prob. o£ Correct 




k ~ i 


k = \ 


A = 10 


Selection 










0.99 


2.7060 


3.2712 


3.6043 


4.5063 


0.95 


1.3528 


1.8362 


2.1261 


2.9210 


0.90 


0.8212 


1 .2434 


1.5026 


2.2244 


0.85 


0.5371 


0.9099 


1.1449 


1.7965+ 


0.80 


0.3541 


0.6826 


0.8961 


1.5139 


0.75 


0.2275- 


0,5139 


0.7074 


1.2811 


0.70 


0.1375- 


0.3832 


0.5575- 


1.0892 


0.65 


0.0742 


0.2792 


0.4347 


0.9256 


0.60 


0.0321 


0.1959 


0.3325- 


0.7832 


0.55 


0.0079 


0.1294 


0.2468 


0.6569 


0.50 


0.0000 


0.0774 


0.1751 


0.5438 
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order of magnitude of the error in our large sample calculations. For ex- 
ample, if h- = 4:,(i* = 0.05 and P* = 0.90, then from Table V we find 
that B = 1.502G and the two expressions in (8) yield 599.54 and 601. 0-t. 
Hence, it would follow from (9) that /; is 000 or 601 or 602. Based on an 
invcKtigation of the hehavior of these two approximations in the case of 
smaller P* or largei' f/* values, it is estimated that the true value of n is 
601 . Even if the correct value is 600 or 002 the error would be less than | 
of I per cent. Fig. ilUistrates these bounds on the Pes for A- = 4, 
P* = 0.50, 0.75 and 0.99. For 7^* g 0.60 the straight line approxima- 
tion is a closer iowei' i)ound than the normal approximation. 

It is estimated that all integer entries in Tables I through IV have an 
error of at most 1 per cent and, in particular, that all entries under 100 
are exact. 

OTHEK VALUKS OF k 

In addition to the tables and giaphs for /.■ = 2, 3, 4 and 10 there are 
also graphs (Figs. 7 through 14) on which interpolation can be carried 
out for h = 5 through 9 and on which extrapolation can be carried out 
for k = 11 through 100. By plotting n versus log /.■ (or n versus /; on 
semi4og paper) and drawing a straight (dashed) line through the values 
of n for // ^ 4 and /,■ = 10 we obtain results which are remarkably good 
approximations for /.■ > 10. The solid cur^■e in these figures connects the 
true values obtained for A- = 2, 3, 4 and 10. 

For large \'alues of k the theoretical justification for a straight line 
approximation is given in Appendix V. In order to check the accuracy of 
our procedure of drawing the straight line through the \'alues of n 
computed for k ^ 4 and /; = 10, we have chosen two points at k — 101 
for an independent computation of the probability of a correct selection. 
For P^ - 0,90, d* = 0.10 and /,■ - 101 the dashed line in Fig. 12 gives n 
as approximately 400. To check this we computed the normal approxi- 
mation to the probability Pf's of a correct .selection for the least favor- 
able configuration in the form 

Pf'-.s ^ r F''\x -h h)J{x) dx = A=r F"^{xV2 + h)c''' dx (10) 

where 

/,, = ^fl^^i^ (- 4.02015 in this example) (11) 

V 1 — d*- 

f(x) is the normal density and F(x} is its c.d.f. This was computed Ijy a 

method suggested by Salzer, Zucker and C'apuano^ and the result was 
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Fig. 7 — Number of units u required per process to gumantee a probability of 
P* = 0.50 of selecting the best of k binomial processes when the true difference 
Pill ~ PI21 is !it. least d". 
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Fig. 8 — Number of units n required per process to guarantee a probability of 
p* = 0.60 of selecting the best of fc binomial processes when the true difference 
V\\\ ~ Ph) is ^^ least d*. 
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Fig. 9 — Number of units n required per process to guiirantee a probability of 
P* = 0.75 of selecting the best of A- binomial processes when the true difference 
Pin — ^[21 is Jit least d*. 
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Fig. 10 — Number of units n required per process to guarantee a probability of 
P* = 0.80 of selecting the best of k binomial processes when the true difference 
Pill ~ P(ai is at least d*. 
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Fig. 11. — Number of units /;. required per process to guarantee a prohiibility 
of p* = 0.85 of selecting the l>eHt of k binomial processes when llic true dirier- 
ence p|,| — pn] is at least d*. 
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Fig. 12 — Number of units n required per process to guarantee a probability of 
P* = 0.90 of selecting the best of fc binomial proeeases when the true difference 
Pill — P(2l is at least </*. 
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Fig. 13 — Number of units n required per process to guarantee a probability of 
P* = 0.95 of selecting the best of h binomial processes when the true difference 

Plu — Pifi is at least rf*. 



1400 






t 






1 




EXACT 

APPROXIMATION 


1000 

aoo 
n 


• 
• 


'/ 
























• 




















































^ 


--■ 


400 












1 


o.^o_ - 














^^,— - 


t^ 


-^ 








0.15 _, 






- — ' 


._ 












— ■ 


— ■ 


— 


-J 


"^ 


0.20 

0.25 ~ 








■-■ 


— 











: i . ■ 



3 4 5 6 8 10 



20 30 40 50 60 80 100 



Fig. 14 — Number of units n required per process to guarantee a probability of 
P* = 0.99 of selecting the best of k binomial processes when the true difference 
Phi ~ Plil is lit least rf*. 
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Pes ^ 0.9168 as compared to the value 0.90 in Fig. 12. The expression 
(10) is derived in Appendix III. Another check was made at P* = 0.99, 
d* = 0.20 and k = 101. The value of n from Fig. 14 is 162. The value of 
the Pes computed from (10) using Salzer, Zucker and Capuano^ is 
0.9925+ . 

Further calculation using (10) yielded the more accurate results 378 
and 154 instead of 400 and 162, respectively, in the above illustrations. 
The error in both eases is less than per cent; for smaller values of Jc the 
percentage error will, of course, be much less. 

For interpolation the results arc estimated to be within 1 per cent 
of the correct value. For example we estimate from Fig. 11 that the re- 
quired value of n for k = 5, P* = 0.85 and d* = 0.05 is 523. This value 
was computed by the normal approximation and found to be 522. 

TIED POPULATIONS 

In computing the tables and graphs it was assumed that if two or more 
populations are tied for first place then one of these is selected by a chance 
device which assigns equal probability to each of them. The experi- 
menter may want to select one of these contenders for first place by 
economic or other considerations. In most practical problems we may 
assume that such a selection is at random as far as the probability of a 
correct selection is concerned. Hence, it appears reasonable to use the 
tables in this paper without any corrections even when the rule for tied 
populations is altered in the manner described above. 

It is interesting to note that in the yield problem the experimenter 
may settle the ((uestion of ties for first place by taking more observa- 
tions until the tie is broken. However, in the life-testing problem he may 
not settle ties by lettuig the test run beyond time T since the best process 
for time T is not necessarily the best for a time greater than T. 

In some applications when there are two or more populations tied for 
first place, the experimenter may prefer to recommend all these con- 
tenders for first place rather than select one of them by a chance device. 
In this case we shall agree to call the selection a correct one if the recom- 
mended set contains the best population (or, when p,,] = piai , if the 
recommended set contains at least one of the best populations). Exact 
tables for the procedure so altered have not been computed. However, 
if the value of 7i is large and this rule for tied populations is used, then 
the experimenter may reduce the tabled values by an amount equal to 
the largest integer contained in \/d*. For example, using the above rule 
for tied populations for the case /; ^ 2, P* = 0.99, d* - 0.30, the tabled 
value 29 can be reduced by 3 giving the result 26. 
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ALTERNATIVE SPECIFICATION 



If the experimenter has some h priori knowledge about the processes, 
then he will prefer to specify the following three quantities in order to 
determine the number n of observations he is required to take from 
each process. 



Specification: He specifies /)*] and p*, (0 ^ pfaj ^ 
pfi] ^ 1) i'l the neighborhood of his estimate of tlie 
probabilities associated w"ith his processes. He also speci- 
fies a probability P* (0 ^ 7'* < 1) that he would like 
to guarantee of making a correct selection ^vhenc\'er the 
true pii] ^ p*i] and the truep(ai ^ p*2] ■ 



(12) 



Table VI — Number oe TTnits Required per PnocES.s to Guaran- 
tee A Probability of P* of selecting the Better ok Two Bi- 
nomial Puocesser when the True p[i) £ pfi] and the True 
Pm ^ P[2] ■ (Alternative Specification, A; = 2) 



i" 


P(,| = (1-75 


^1,1 = o.yi 


p\,] - 0.90 


pU = 0.85 


PU\ = 0.95 




p],l = (l.dG 


p'li] = 0.80 


y.|,, - 0.80 


f'-l = 0.80 


f ',j = 0.90 


0.50 


1 


1 


1 


1 


1 


0.60 


2 


2 


;i 


9 


6 


0.75 


10 


6 


13 


53 


27 


0.80 


14 


s 


10 


83 


40 


O.So 


21 


11 


2S 


124 


60 


().!)0 


32 


Ifi 


42 


189 


91 


0.95 


53 


25 


68 


312 


149 


((.99 


106 


49 


135 


623 


298 



Table VII — Number of Units Required per Process to Guar- 
antee A Probability of P* of Selecting the Best of Four Bi- 
nomial Processes when the True p^^^ g p*] and the True 

Pm ^ P[2] ■ (Alternative Specification, /.- = 4) 



P' 


P[,\ = 0,7S 


P'u] = 0.95 


P'U] = 0,90 


p', 1 = 0,S.=i 


Jl III = 0.95 




il>f,| = O.fiO 


pjil = 0,80 


p'[-_] = 0,HO 


p'..^ = 0.80 


/if.;] = 0.91) 


0,50 


7 


4 


10 


42 


21 


O.tJO 


14 


S 


18 


79 


39 


0.75 


2S 


14 


37 


168 


80 


0,80 


35 


18 


46 


211 


101 


0.85 


45 


22 


59 


268 


128 


0.90 


59 


28 


77 


350 


171 


0.95 


83 


39 


107 


493 


239 


0,99 


139 


65 


1S2 


831 


399 
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Again we can rewrite the specification that the experimenter wants 
to satisfy in the simple form 



Cfl 



^ P* for p[ij ^ pui and pi2] ^ Pm 



(13) 



Tables VI and VII give the number of observations required per proc- 
ess for several selected triplets of specified constants (pdj , p[-i] , P*) 
when k = 2 and /c = 4. These results are also given in graphical form 
in Figs. 15 and 16. 




Fig. 15 — Number of units required per process to guarantee a probability of 
P* of selecting the better of two binomial processes when the true p m S P ui and 
the true piJi ^ pn] . 
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For example, on the basis of past experience the experimenter may 
estimat4; that the probabilities associated with his /; = 4 processes are 
ail in the neighborhood of ().(jO. Thi« constitutes his ji, priori knowledge. 
He may then decide that he would like to make a correct selection 
with probability P* = 0.85 when the best pi-ocess has a yield of at least 
7o per cent and all the othcr.s ha\"e a y\old of at most GO per cent. Enter- 
ing column 1 of Ta!)le \'II \vc find that /( = 45 observations per process 
are required. 

It is much more difficult to furnish tables for the alternative specifica- 
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Fij;. 16 — Niiiiil>cr of units reiiuired per (jrocess to guarantee a probability of 

P* of selecting the best of four I>inomial i)rocosses when the true pn] g p[,| and 
the true ;j[2| ^ 7J[2] ■ 
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tion since there is an extra parameter to vary and the appropriate 
tables for the normal approximation are not available. 

In the computation of these probabilities the least favorable con- 
figuration 



p[i] = Pui and p*2i = p[2] = P[3] = ■ • • - Pit] 



(H) 



was used. It follows from Appendix II that if the probabiUty of a correct 
selection is at least P* when (14) holds, then it will also be at least P* 
when 

Vii] ^ P*i] and p*2] ^ p[2] ^ Pw ^ ^ Pw (15) 

For small values of ?i, exact calculations were carried out. A typical 
exact calculation is shown in Appendix IV. The approximations used 
for large n are given in Appendix III. 
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Fig 17 — Illustration of the varying zones of indifference and the least favor- 
able configuriLtion for A. = 4 and P* = 0.85. (For n ^ 5 the longest vertical segment 
occurs at the point A„ where the abcissa and the ordinate are symmetrical about 
0.5) 
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COMPARISON OF THE TWO SPECIFICATIONS 



It should be pointed out that for a given k the same value of n would 
satisfy the specification for different specified triplets 



P\ P*i 



Pm 



For example with A- = 4, P* = 0.85 and n fixed we could vary p*i] in 
the alternative specification and compute for each p[i] the correspond- 
ing largest vahie of p*2i such that the specification (P*, pm , pi2\) is 
satisfied. This is shown in Fig. 17 for n = 5, 10, 20 and 60. The vertical 
distance in Fig. 17 between the appropriate curve and the 45° line 
(n = »:) is the length of the indifi'erence zone (p*] , P[2]). The indif- 
ference zone widens in the center and narrows at both ends. In fact we 
find just as in the original specification that for n greater than (say) 4 
the indifference zone is widest when ?>*] and p*2] are symmetrical about 
0.5. It is clear that the two specifications would coincide if we took d* in 
the original specification and set pm ^ ^ (1 + d*), p[2] = ^ (1 — d*) 
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Fig. 18 — Probiibility of a correct selection aa a function of the true difference 
^ = Pill ~ V\".\ under tlie least favorable contiguratioii for k = 2 and selected 
values of n. 
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in the alternative specification. We shall be interested in comparing the 
alternative specification (P*, p*] , p*2i) with the original specification 
with the same P* and with d* set equal to p*i] - p*2] . It is clear that 
the value of n rctiuired for the original specification will always be 
larger. 

The original specification is simpler and is preferable to the alternative 
specification when little or nothing is known about the processes on test, 
but the price that has to be paid for ignorance is an increase in the 
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Fig. 19 — Probability of a correct selection as a function of the true difference 
^ = pi^j _ pij, under the least favorable configuration for fc = 3 and selected 
values of n. 
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required number of observations. In the example of the preceding sec- 
tion the value of n required for the alternative specification is 45 as 
compared to 51 observations per process required to satisfy the original 
specification with the same P* and with d* = p*,j — pfz]. Here the 
saving is only moderate. The saving will be much larger if p*i] and 
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Fig. 20 — ■ Probitbility of ;i correct selection as a function of the true difference 
^ = V\\\ ~ VVi\ under the least favoriil)le configuratiim for A' = 4 and selected 
values of n. 
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p*2] are further from 0.5 and d* is small. For example, for A = 4, P* = 
0.95, pfu = 0.95 and p*2] = 0.90 the value of n required for the alter- 
native specification is 239 as compared to 850 observations per process 
required to satisfy the original specification with the same P* and with 
d* = 0.05. The alternative specification is justified on the basis of k priori 
or previous information about the approximate values of the p's. 

BEVERSING THE TABLES 

The experimenter may wish to use the tables of this paper in reverse. 
For example, if n is fixed and d* is specified by the experimenter, then 
by using the appropriate table he can find the probability of a correct 
selection that is guaranteed for d ^ d*; i.e., a greatest lower bound to 
the probability of a correct selection for d ^ d*. This process of re- 
versing the given values and the values to be computed can most easily 
be carried out on graphs. For example, the above problem of finding the 
guaranteed probability of a correct selection given d* and n is most easily 
carried out on Figs. 18, 19, and 20. 

Appendix I 

MODIFICATION OF THE ORIGINAL SPECIFICATION 

The same value of n will, of course, satisfy the specification for dif- 
ferent pairs of specified values (rf*, P*). From a purely mathematical 
point of view it is not necessary that d* should be the smallest difference 
for which the experimenter desires to make a correct selection. For ex- 
ample, if /c = 3 the experimenter could specify any one of the four pairs 
(0.10, 0.60), (0.25, 0.90), (0.30, 0.95) or (0.40, 0.99) and obtain the same 
result, namely n = 20. The experimenter may prefer to specify the curve 
or set of points corresponding to a fixed n. Several such curves are given 
in Figs. 18, 19, and 20 for k = 2, 3, and 4, respectively. The experi- 
menter would decide in advance on some property of the curve that he 
considers desirable and from the appropriate figure he could find the 
curve with the smallest n-value that satisfies the desired property. 

The main point of the above paragraph is to point out that the original 
specification in the body of the paper is one particular way, but not the 
only way, of stating a specification that will determine a value of n. 
The only criterion for a good way to state the specification is that the 
experimenter should Ije able to bring his best judgment (or best guesses) 
to bear on the quantities that have to be specified in advance. 
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Appendix II 

MONQTONICITY PROPERTIES 

AVe shall prove that for any fixed d {0 ^ d ^ 1) the probability Pes 
of a correct selection is smaller for the configuration: 

Pli] - d ^ ])\^] - /j(3] - ■ ■ ■ = p[i., (Al) 

than for any configuration gi\'en liy 

/J[ii - d ^ pi->] ^ p[3i ^ ■•■ ^ p[^] (A2) 

where p|i] is considered fixed and the p(,| (i ^ 2) are variables. In other 
words, for fixed p[i\ the probability Pes is a strictly increasing function 
of each of the differences 

Pill - Pbi (i ^ 2) 

We shall need the following lemma. 

Lemma 1: For any pair of integers ;r, n (0 g .-c ^ ti) and any 
6 {0 ^ 6 S 1), not depending on p, the function 

j:-1 

H(x; p, e) = E C/'p'(\ - p)"-' + ffCVp'd - p)"-' (A3) 

is a decreasing fi<nr(ion of p over the unit interval (0 ^ p ^ 1). More- 
over, it is strictly decreasing unless (.r ^ and 6 = 0) or (x = n and 

e= 1). 

Proof: Differentiating (A3) with respect to p gi\es after telescoping 
terms 

(6 - l)a-r."/-Vl - p)"-' - ein - .r)C'."/(l - p)-'"' (A4) 

which is negative for < /) < 1 unless (.c = and 6 = 0) or {x = n and 
d = 1). Since// is continuous in p oXp = and p = 1 the lemma follows. 
Let X(o denote the chance number of successes that arises from the 
binomial process associaterl with 

P\i] {i = 1,2, ■■• , 71.) 

the value of the integer n is assumed to be fixed throughout this discus- 
sion and it will usually not be listed as an argument. The probability 
Pes of a correct selection for any configuration with p[i] > p[2i is 
given by the expression on the top of the next page. 
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ft 

Pes = P{X(o < Xd) for i^2] -\-h E Pf^M = X(i, and 

Xco < X<i) for I ^ 2, i" ?^ q} + • ■ • CA5) 

+ ^P{X<i) = X(2) = ... = Zu)} 

It will be necessary to write the Pes for any configuration with p[i] > p[2] 
in another form which is more usefxil for the purpose at hand. Corre- 
sponding to any binomial chance variable X (which takes on integer 
values from to n) we define a "Continuous Binomial" chance variable 
V by letting Y be uniformly distributed in the interval 0' - |» J + 2) 
with the same total probability in this interval as the ordinary binomial 
assigns to the integer j, namely 

CVa -P)"~' (3 = 0,1, ••-,») 

We will now show that the probability Pes of a correct selection is unal- 
tered if v/e replace each of the k discrete binomials by its corresponding 
continuous binomial. Let Y^i) denote the continuous binomial (CB) 
chance variable associated with p[.| and let ?/(,) denote any value it can 
take on. Let X(o denote the nearest integer to 7(0 and let .i;{o denote 
the nearest integer to y^t) (i = 1,2, ■■■ , k). Then X(o is a discrete bi- 
nomial (DB) with the same parameters (pn] , n). Let 

g(x, v) = C.>^(1 - VY" (x = 0, 1, • • ■, n) 

Then the density g(y, p) of the continuous binomial (disregarding the 
half-integers) is given by giy, p) - gix, p) where x is the nearest integer 

For two continuous binomials (i.e., k = 2) the probability Pes of a 
correct selection for any configuration with pm > pm is given by 

PcsiCB) = / P{Yc2) < ym\g(uw ; Pui) d'Jo) (^6) 

J- 1/2 

n rxiD + lli 

= E / P{Yi2) <y(i)]g(y(i);pii\)dy(i) (A7) 

Within any interval (aid) - 7, :C(n + I) we have 

P{yw < yo)] = -P{Xc2i < xa)] ■ ^^g^ 

-i- P1X(2) = xa)]P{Ya) < Vw 1 Xc» = x^)] 
= P{X{2) < xa)] + i P{Xm = xw\ (A9) 

which depends only on x^d . Hence from (A?) 
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PcsWB) 

" (AlO) 

= P{X«) < Xu)} + iP{X(2, = Xa,} (All) 

= Pc5(/)B) (A12) 

The above is easily generalized to hold for any k > 2. The details of this 
generalization are omitted. For general k this equality holds not only 
for the important special case p^ > p[2] but also for the more general 
case (2) for any t < k. Since the latter result is not needed here, the 
proof is omitted. 

If wo let G(!j;p) denote the c.d.f. of the continuous binomial then 
lemma 1 can be restated in the following form. 

Lemma 2: For any integer n and any y, the function G{y\'p) is a non- 
increasing junction of p. In particular, for —^ < y < « + | it is a 
strictly decreasing function of p. 

Proof: For any y, set x ^ .T(y) and 9 — e(y) equal to the integer part 
and the fractional part of (// + |), respectively. Then for any y we have 
the identity in p 

G(y;v) - H(:v;p, d) (U ^ p ^ 1} (AI3) 

For any ^o such that —^<yu<n-\-^ we have ^ x{yn) ^ k and 
^ 6((/o) g 1. The inverse function y(x,6) = x -\- 6 — ^ is a single- 
valued function of the pair {x,6); the two particular pairs (0,0) and 
{n,l) correspond to the unique \'alues y = — | and y = 7i + ^, re- 
spectively. Hence the pair {x(yo), Oiyo)] must be different from these two 
particular pairs abo^e since it corresponds only to //o which is in the 
interior of the interval ( — |, // + ^). Lemma 2 then follows from lemma 1 
and the fact that G{y\p) is identically zero in p for /y g —\ and identi- 
cally one in p for y '^ n -\- \. 

The probability Pes of a correct selection for k discrete or A- continuous 
binomials for any configuration with p\\] > p|2] can now be written as 



J-l/2 



n + il2 r t 



II ^{5^(0 <^(i.} 



gOjd) ; Pui) i^ua) (AU) 



= f \tLC^(>/;Pud\9(r,Piii) d'j. (A15) 

Clearly if any one or more of the pif] (i ^ 2) decreases, holding pm 
fixed, then it follows from lemma 2 that the right member of (A15) is 
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strictly increasing, i.e., for fixed pm the Pea is a strictly increasing func- 
tion of each of the differences p[i] - pin (i ^ 2) as was to be shown. 

It follows from the above result that in searching for a least favorable 
configuration among all those in which the experimenter wants his 
specification satisfied we may restrict our attention to those of the form 
(Al). Moreover, we may set d in (Al) equal to d* since, for d > d* and 
fixed p[i] , the difference d - d^ may be added to each pn] (i ^ 2) and 
the probability of a correct selection is increased. Then (A15) reduces to 

F^, = r""' ' G'"\y;pii^ - d*)g{y;vm) dy (A16) 

It was shown in the section on the least favorable configuration that 
there is a value pf'i] of pm which when substituted in (A16) gives the 
minimum value Pes of Pcb • 

We can now prove the following result in which p[i] is not fixed. For 
any specified pairpf2] ^ pfn the probabiUty Pea of a correct selection 
is smaller for the configuration 

p[i] = pfii ; P*2] = Pm = Pm = ■■• = pw (A17) 

than for any configuration given by 

p[i] ^ P*i] ; Ph ^ Pm ^ Pw ^ ••■ ^ P[*i (-A-IS) 

This is shown by considering two separate steps. 

The first step is to increase p[i] holding all the other p's fixed at p[2] . 
For any arbitrary set of values of pn] with pn] > pm the probability 
of a correct selection can be written as 

Pas =111 n G{y,p,A) [1 - G{y,p[n)]g{y,pu{) dy (A19) 

,=2 ''"1/2 Li-2. i^i J 

by adding the probabilities that 

1^(1) > Yu) > min { F(2) , • ■ ■ , F(,_i) , Yu+d , ••• , Yik) 1 
for 

y = 2, 3, • • ■ , fc 
For 

Pm > Pm = P[3! = ■ ■ • = p[k] = p{2] 
this reduces to 

P,, = (/c - 1) /■""'"' [1 - G{r,p,^,)]G''-\y;piMy,pt2^) dy (A20) 

J-l/2 
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This resuK. can also be obtained by starting with (A16) with 

and integrating by parts. It is clear from (A20) that for fixed p[,-i (i ^ 2) 
the Pes is an increasing function of p[i] and is indeed strictly increasing 
for p[i] in the unit interval. 

The second step is to hold pm fixed and to decrease the values of 
P[i] (« ^ 2). This increases the probability of a correct selection by our 
previous result above. This proves the nionotonicity property for the 
alternative specification. 

Appendix III 

LARGE SAMPLE THEORY — ORIGINAL SPECIFICATION 

For p[i] > p[2i the probabihty of a correct selection satisfies the in- 
equalities 

P{Xa) > XciAi = 2,3, ■■■ ,k)] < Pes 

< P{Xa) ^ X(,)(t = 2,3, ■•■ ,fc)l (A21) 

unless pii] = 1 and Pi2] = in which case equahty signs hold since the 
three quantities above are all unity. Lotting q^ = 1 — p^ , we can 
write the left member of (A21) as 

pk> , -^*^^ (^ ^1,2, •■■,/. -i)\ 

^ Vp[i]?[il + {pw - f/*)(9iii + d*) J 

(A22) 

where 

' ^ Vn[p„5[, + (pm -rn(9„+r/*)] (' = 1. 2, . . . , fc - 1) 

(A23) 

For the configuration (Al) with d = d* the chance variables Zi tend to 
normal chance variables A^(0,1) with zero mean and unit variance as 
n ^ ^ . We have purposely omitted any continuity correction in (A22) 
in order to get a better approximation for the smaller values of n. 

To derive the least favorable configuration for large n we can^estrict 
our attention to those configurations given by (Al) with d = d*. The 
quantity pj'i] , which minimizes (A22), is obtained by maximizing the 
expression in (A22) 

Q(p) = p(l - p) + (p - d*)(l -p + d*) (A24) 

= ~2p' + 2(1 + d*)p - d*(l + d*) (A25) 
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The derivative of Q{p) vanishes at 

P^] = §(l + rf*); gm = H1 ~d*) CA26) 

which gives the symmetric configuration. Clearly this value of p gives 
to Q{p) its maximum value, HI ^ '^*^)- This proves that the symmetri- 
cal configuration is least favorable in the limit as n ^ -» . 

Under the configuration (Al) with d = d* and n -^ =c the distribution 
of the chance variables Zi (i = 1,2, • • - , A- - 1) approaches a joint 
multivariate normal distribution with zero means, unit variances and 
correlations given by 

which do not depend on n. For the symmetric configuration this reduces 
to the simple form 

p(Z,-Z,) - I (t 9^3)- (A28) 

This is precisely the case which arises in [1] and consequently the tables 
in [1] can be used for our problem when (the answer) n is large. The con- 
stants C = C{P*, k) tabulated in [1] solve the equation 

P[z,> -:^ii= 1,2, ■■.,fc-l)}=P* (A29) 

for standard normal chance variables Zi satisfying (A28). If we equate 
C/\/2 and the corresponding member of (A22), then we obtain for the 
symmetric configuration 

_g_^ d*V7i 

V2 — A (A30) 

or solving for n and letting B = \C'^ this yields the large sample normal 
approximation 

n^^,(l-d*^) (A31) 

Since d* is usually small when n is large and since the solution in (A31) is 
usuallj* somewhat smaller than the true value, then it is of interest to 
examine the simpler approximation 

n^A (A32) 

which is greater than the result in (A31). This is called the straight line 
approximation since it plots as a straight line on log-log paper as shown 
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in Figs. 2 through 5. As d* -^ both the normal approximation and the 
true value are asymptotically equi\'alent to the straight line approxima- 
tion. 

The normal approximation to the probaliility of a correct selection 
can also be written in another form similar to (A16) which is actually 
more useful for numerical calculations. The left member of (A21) can 
be written as 



E [n p { 



(A33) 

Vp[i]qii] 



< ^^'^1 Vp[i]g[ii + (Pm - Pii]) V^'A I p n^^ = y^i 



where 



and Mil is the same function of .r<i) as IFi is of X(i) . The outside summa- 
tion in (A33) is over the values taken on by ivi as X(i) runs from to n. 
As n — > «> the expression in (A33) approaches 

Pe. ^ r Tn F f ^ V^^i^ +_^- P"-^^ ^^ ^)1 / iw) d,o CA35) 

-'-« Li=5 \ Vp[i](l[i] /J 

where /(O is the standard normal density and F(t) is the standard normal 
c.d.f. For the symmetric configuration, which is least favorable for large 
n, (A3o) reduces to 

P'cs ^ r F'-' (iv + T"^"' Vm '^^ (A36) 

A straightforward integration by parts gives the alternative form 



....-:, i:[:-.(.-^:^) 



F'-\iv)f(w) dw {A37) 



which corresponds to (A20). 

A simple method for computing such integrals based on Hermite poly- 
nomials is described bj' Salzer, Zueker, and Capuano.^ 

LARGE SAMPLE THEORY — ALTERNATIVE SPECIFICATION 

The expression corresponding to f A22) for the alternative specification 



IS 



PiZ<>-^/f\-f;^l^; (:i^,,2,---,k-l)} (A38) 

Vpf^qfii +F?>r*2i9*2i 
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which is ah-eady written for the least favorable configuration. The tables^ 
are not immediately appUcable since the correlations 

»(ZX,) = . f''f\ , - (i ^3) (A39) 

are not, in general, equal to \. In the cases treated in Tables VI and VII, 
p*2i ^ 0.5 and hence p*i]?*i < P*2!?*2! ^o that the correlations (A39) 
are all less than ^. It was found that linear interpolation on the required 
value of n between the results for p - and p = \ gives moderately 
good results when n is large. The result for p = | is given by 

(P[I] - P[2l) 

with \ = 2B where B is given in Table V. The result f or p = is given 
by (A40) with X = Xo' where Xo is the solution of the equation 

P{Z > -Xn) ^ P*""'"" (A41) 

which can easily be found from univariate normal probability tables. 
An explicit expression for the result of this linear interpolation is 



^ Pm9ui(-^ - ^o') + vti]Qh^ _^ /foi- p*. > 0.5) (A42) 

(P[l] - P[2]J 



The expressions for the probability of a correct selection for the al- 
ternative specification corresponding to (A36) and (A37) are 

P^.^ ^ r F'-' (aw + b)fiw) dw (A43) 

J— 00 

= a{k - 1) [ [l - f ('^)] P'-\w)j{w) dw (A44) 
where 

-= 4/44 > and b = '^J^^^^pp^ > (A45) 

These expressions can also be evaluated by the method described by 
Salzer, Zucker and Capuano.'' 
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Appendix IV 

TYPICAL EXACT CALCULATION 

A. Original Spccificalinn 

The exact expression (A5) for the probability of a correet selection 
for any configuration simplifies if the configuration is least favorable. 
For any pair of integers {j,n) we define 

b,j = P[X<i) = j] = C/(p[-„)\ff^j)""' (0 ^ y ^ n) (A46) 

hj = P{Z(„ = it = C'/Cpfu ~ d*y(gl\^ + d*r-^ (O^jS n) (A47) 

B,j = P{X,,, gil (A48) 

Then the exact probability Pc.^ of a correct selection for the least 
favorable configuration can be written as 

Pes = E hj E -4:1 6*. 5*Ti-' (A49) 

where ^2,-1 is defined to be zero. Here, for each value of X(i) , the letter 
i denotes the number of processes tliat tie with A'(i) for first place and 
for any gi\'en \alue of i the conditional probability of a correct selection 
is 1/(1 + i). Taking /; = 4 as a typical case, we can write (A49) more 
explicitly as 

Pas = E hjBij.^ + ? E h,jh,jBij.^ + E hjhhB2j-i 

(A50) 

+ , E &iy^2j 
4 ,=0 

If n ^ 10 then we may use the symmetric configuration, i.e., we may set 

P[i] = 3 (1 + <-f*), ill computing from (A49) or (A50). 

B. Alternative S-pecification 

The probability P'is of a correct selection for the alternative specifi- 
cation is the same as in (A49) and (A50) except that we now define 

hii = P[X,., = j\ = C/\vt<,y(qhy~\i = 1, 2) (A51) 

B2i = P{Xr.) ^j] (A52) 

A typical exact calculation for /,■ = 4, using (A50), (A51) and (A52) with 

Pui = P*i\ = 0.7o 
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and 

p[2i = Vm = Pw = Pw 

is given in Table AI. Exact values for the individual and (cumulative 
binomial probabilities were obtained from References 4, 5 and 6. 



Table AI — Calculation of the Pes 



p[ii = P*i! ^ 7c 


; P[2 


= P[3] 


= Pii] = 


P[i] = 


0.60; 


k = 4: 


J 


6'.j 


b2.i 


Bi.j.i 


bli 


H.i 


Bli.i 


B3.M 





0.00000 


0.00010 


. 


0.00000 


0.00000 


— 


— 


1 


0.00003 


0.00157 


O.OOOIO 


0.00000 


0.00000 


0.00000 


0.00000 


2 


0.00039 


0.01062 


0.00168 


0.00011 


0.00000 


0.00000 


0.00000 


3 


0.00309 


0.04247 


0.01229 


0.00180 


0.00008 


0.00015 


0.00000 


4 


0.01622 


0.11148 


0.05476 


0.01243 


0.00139 


0.00300 


0.00016 


5 


0.05840 


0.20066 


0.16624 


0.04026 


0.00808 


0.02764 


0.00459 


6 


. 14600 


0,25082 


0.36690 


0.06291 


0.01578 


0.13462 


0.04939 


7 


0.25028 


0.21499 


0.61772 


0.04622 


0.00994 


0.38158 


0.23571 


8 


0.28157 


0.12093 


0.83271 


0.01462 


0.00177 


0.69341 


0.57741 


9 


0.18771 


0.04031 


0.95364 


0.00162 


0.00007 


0.90943 


0.86727 


10 


0.05631 


0.00605 


0.99395 


0.00004 


0.00000 


0.98794 


0.98196 


Check 
















totals .... 


1.00000 


1.00000 













Z &!.; -Bli-i = 0.44715 



j-i 



tJlKiKiBh-i = 0.08493 
J ,=1 

10 

T,KihliB2.,-i = 0.01464 



;=i 



1 ^° 

^T^byj^l^ = 0.00145 
4 j=o 



Total - 0.54817 - Pes 



Appendix V 

In this appendix it will be shown that for large values of k the value of 
n requu-ed to meet any fixed specification (d*, P*) is approximately 
equal to some constant multiple of (In k) . 

Let n = n(k) denote the unique positive decimal solution of the equa- 
tion 






F'-\w + h Vn)f(w) dw = P* 



(A53) 
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wiierc f{w) and F{w) are defined above, P* and h are known constants 
with 1/k < P* < 1 and /? > and the argument k is a positive integer. 
Let e be a (small) fixed number such that < £ < Min (P*, 1 - P*). 
Then e < P* —]//i; for sufficiently large A. Let ,1 = ,l(f) be defined by 



/_' fiw) 



dw ^ 1 - € (A54) 

J— A 

SO that 

< f 7^^"'(w + h \/^i)f(iv) dw ^£ {A55) 



for any integer k ^ 1, any ?i > and any h > 0. Let n' and 7t" be the 
uni(|ue positive decimal solutions, respecti\'e!y, of the equations 

f F'^'iw + h V^Ofiw) dw = P* - £ (A56) 

J-A 

f F''\w + h V^')fiw) dw = P* (A57) 

J— A 

where P*, b and k are the same as in (A53). It follows from (A55), (A5G) 
an<i (A57) that for any integer /.■ ^ 1 

n' ^ II ^ n" (A58) 

From (A54) and (A57) we have 

[ F'-\io + b-y/^')fi(w) dw = -^ (A59) 

J~A ' 1 — £ 

where f.\(w) is the density of the normal distribution, truncated at A 
and ~A. The right hand memljer of (A59) is positive and less than 
unitysincef < 1 — P*. Hence there exists a ir_4 with |«'.4 | ^ Asuchthat 

r F'~\w + b^/J^')fAw) dw = F''\wa + h>/^') (AfiO) 

Since iva is bounded and ?i" is large for large /: we can use the \vell-kno^\■n 
approximation 



F^-Kw^^W^)=\l~'-^^^ 



+ hVn") J ^^ ^ 

,_, , (A61) 
_ , (fc - 1) exp [-(w^ + bVn"y/2\ 



V'2Tr{wA + h\/n") 
where only the leading term is considered. Hence from (A59)j (AGO) 
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and (A61) 



(A62) 

^ Kwa + h\/n'~'f + In {wa + bVn^') 

Since Wa is bounded and In V^ ^ o("^") i*^ follows that for large k 

n" ^ (2/6') In (k - I) ^ C In /^ (A63) 

where C is a proportionality factor. Starting with (A54) and (A56) the 
same argument gives the same result as (A63) for n'. Hence, by (A58), 
the same result must hold for n. 
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